1. Field of the Invention
This invention relates to magnification of touchscreen displays. More specifically, it relates to software and methods to improve navigation of a touchscreen graphic user interface (GUI) while under magnification.
2. Brief Description of the Related Art
As technology has advanced, graphic displays have vastly improved in clarity and resolution. Not many years ago, 15 inch monitors displayed GUIs at VGA resolutions (640×480 pixels). However, portable, touchscreen displays like those sold under the KINDLE FIRE HDX brand provide resolutions on an 8.9 inch screen of 2560×1600 pixels or 339 pixels per inch density. The ability to present fonts, images and icons clearly at high resolutions has provided opportunities for interface and software designers to put more content on a single screen. However, as users move to touchscreen devices the screens themselves have become even smaller.
In conjunction with higher screen resolutions in smaller displays, another concurrent phenomenon is the method of navigation itself. From the 1980s to 2010 the most common tools for navigating a GUI were keyboard and mouse peripherals. In the most popular operating systems, icons were placed on a static desktop canvas defined by the resolution and size of the physical display monitor. By addition of one or more monitors, the desktop canvas could be expanded to provide more surface area. Nevertheless, this canvas was static and limited the surface area upon which icons and other control objects could be placed. While the desktop canvas could theoretically be made to “scroll” this was not an intuitive feature for users using a keyboard or mouse to navigate about the interface. However, this was to change.
While some touchscreen displays were available in a desktop orientation and others (like Microsoft Corporation's early SURFACE brand technology) operated in a tabletop orientation, these were not portable and did not support optimum ergonomics. When Apple, Inc. introduced the IPAD touchscreen tablets on Apr. 3, 2010, users could position the device in a comfortable orientation to navigate by touch. On the software level, touchscreen navigation is operable by user “gestures.” Initially, these gestures only focused on the Cartesian coordinates of a single touch point on the display and perhaps a single or double-tap on the screen to fire an event on the device. However, as the technology advanced, devices were able to detect “multi-touch” meaning that one, two or three fingers simultaneously touching the screen could signify different operations or states.
From a navigation standpoint, portable touchscreen devices presented challenges but new opportunities. The challenges were a smaller display and smaller desktop in which to show icons and controls. The new opportunities lay in the intuitive nature of sliding a desktop canvas around. Touchscreen devices lend themselves to a new navigation paradigm. For example, an optical microscope will typically only focus on a small area of a slide. As the viewer wants to see other areas of the slide they push the slide with their finger while the viewport of the microscope remains static.
By analogy, the display area of a portable device is like the viewport of the microscope . . . it only can see a small area of the entire desktop canvas. By registering gestures on the touchscreen, the desktop canvas “slides” under the display viewport in an intuitive manner. There are two types of approaches to this movement: (1) scrolling; and (2) paging.
In a scrolling approach the desktop canvas is like a large slide under a microscope and touchscreen gestures like “swipes” scroll the canvas in the direction it is pulled by the swipe gesture. This action is similar to moving a sheet of paper around a desk with a finger.
The swipe gesture may react differently depending on the speed, distance and contact on the finger on the touchscreen. For example, contacting the screen with a single finger and maintaining the finger in contact with the screen while moving the finger to the left scrolls the canvas the same distance to the left as the finger moved. Another gesture is a “flick” in which the finger moves rapidly to the left in which case the canvas continues to scroll to the left even when the finger is lifted off the screen from the flick gesture. The scrolling may have an “inertia” effect wherein the canvas initially scrolls at the speed of the flick gesture but then slows down to a stop as if the canvas is subject to some friction or other resistive force.
Screen magnification software is well-known for traditional displays coupled to desktop and laptop computers. The magnification software may be built into the operating system of the computer or may be third party applications such as those sold under the MAGIC brand by Freedom Scientific, Inc. Screen magnification software on a traditional computer display typically magnifies a portion of the screen at a user-designated magnification level (e.g., 8×). When this happens, the entire canvas of the desktop cannot be displayed because at magnification only a portion of the canvas is shown. This is frequently referred to as the “viewport.” Using mouse or keyboard commands, the user would pan around the canvas (whether the background desktop, over an application in the foreground or the like). For traditional operating systems on personal computers, the boundaries of the canvas were the edges of the physical display monitor (or monitors for multi-monitor configurations). As the viewport panned to the edge of the canvas the user simply came to a hard boundary and there was nothing left to scroll to.
However, as noted above, touch-screen device operating systems frequently provide a canvas of far greater area that what can be displayed at one time. Therefore, under no magnification, the swipe gesture moves different parts of the canvas into the field of view. This becomes a problem when the touch-screen device is using screen magnification. For example, if the user is at 4× magnification, the user can only view a section of what is viewable at 1×. A user may invoke an “explore” mode by holding down a plurality of fingers onto the screen (e.g., three fingers) which moves the magnified view about the boundaries of the display at 1× but does not scroll the canvas beyond the 1× display boundaries. When the user explores up to the 1× display boundary they are required to switch into a “pan” mode to move otherwise hidden canvas into the display area. Pan mode may require change the number of fingers held down on the touchscreen (e.g., one finger) or that the user switch between dedicated touch modes (i.e operating system gestures and magnification program gestures). The user would then resume the explore mode again to view sections of the newly available canvas areas at 4×.
A drawback of the current state of the art is requiring the low-vision user to switch between explore and pan modes when coming up to a viewport boundary. What is needed in the art is a method and/or software product to detect a viewport boundary that contains additional canvas in the same direction and automatically pan the user to new canvas area without leaving the explore mode.
However, in view of the art considered as a whole at the time the present invention was made, it was not obvious to those of ordinary skill in the field of this invention how the shortcomings of the prior art could be overcome.